Orthogonal series and limit theorems for 
canonical U- and ^/-statistics of stationary 
connected observations 

Abstract 

' The limit behavior is studied for the distributions of normahzed U- and V- 

^ ■ statistics of an arbitrary order with canonical (degenerate) kernels, based on 

CsJ ! samples of increasing sizes from a stationary sequence of observations satis- 

5;^ I fying (f- or a-mixing. The corresponding limit distributions are represented 

, ^ ■ as infinite multilinear forms of a centered Gaussian sequence with a known 

covariance matrix. 

keywords: stationary sequence of random variables, mixing, multiple or- 
thogonal series, canonical U- and V-statistics. 
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1 Introduction. Preliminary results 



In the present paper, we study the limit behavior of the distributions of normalized 
canonical U- and l^-statistics based on stationary observations under cp- or a-mixing. 
The approach based on a kernel representation of the statistics under consideration 
^ . as a multiple series, is quite similar to the approach in fTT] where the analogous 

•jQ ! results were obtained for independent observations. First of all, introduce some 

definitions and notions (see [H1I9])- 



Definition 1. Let {X,A} be a measurable space with a measure /i. We say that the 
O ' measure fi has a countable basis if there exists a countable family 

21 := {An, n = 1,2,...} 



^ , of measurable subsets (a countable basis of the measure fi) such that, for every M G 

^ I A and any e > 0, there is A^ E ^ such that 

/i(MAAfc) < €. 

For example, if X is a separable metric space and A is the corresponding Borel 
cr-field then every cr-finite measure on A has a countable basis. 

Let Xi, X2, ... be a stationary sequence of random variables defined on a prob- 
ability space {Q,!F,F) and taking values in an arbitrary measurable space {X, ^}. 
Denote by F the distribution of Xi. In the sequel, we consider only the distributions 
on A which have a countable basis. In addition to the stationary sequence intro- 
duced above, we need an auxiliary sequence {X*} consisting of independent copies 



1 



of Xi. Denote by L2(X'",F'") the space of measurable functions f(ti, . . . ,tm) 
defined on the corresponding Cartesian power of the space {X, A} with the corre- 
sponding product-measure and satisfying the condition Kf'^{Xl, . . . ,X^) < oo. 

Definition 2. A function /(ti, . . . , tm) ^ Iv2(X™, F"*) is called canonical (or degen- 
erate) if 

Ex*fiXl,...,X*J = a.s. (1) 

for every k, where Kx* is the conditional expectation given the random variables 
{X:;t^k}. 

Define a canonical Von Mises statistic by the formula 

K = V;(/) := n--/2 Y: f{X,„...,X,J, (2) 

^<jl,--;jm.<n 

where the function /(ti, . . . ,tm) (the so-called kernel of the statistic) is canonical. 
For independent {^i}, such statistics called canonical V -statistics as well, are stud- 
ied during last sixty years (see the reference and examples of such statistics in [9]). 
For the first time, some limit theorems in the bivariate case were obtained in [TOl [6]. 
In addition to \^-statistics, the so-called ^/-statistics were studied as well: 

[/„ ^ f/„(/) := /2 ^ /(X,„...,X,„) (3) 

or 

Ul:=n--" Y: /o(X,,...,X,„). (4) 

l<il<--- <im<n 

Notice that, in addition, the kernel /o in (jlj) is assumed to be symmetric (or invariant 
with respect to all permutations of the variables). 

The main difference of [/-statistics from ^-statistics is that, in the region of 
summation of the corresponding multiple sums ([3]) and (jl]), the so-called diagonal 
subspaces are absent, i. e., all the subscripts are pairwise distinct. If the distribu- 
tion F of the random variable Xi has no atoms then setting the kernel to be zero on 
all the diagonal subspaces, we can easily reduce [/-statistics in to l^-statistics ([2]). 
Moreover, it is easy to see that formulas ([3]) and (jl]) for [/-statistics are equivalent: 
If in (HD we set 

/o(^l; • • • ; tm) '■= Y fi'^h^ • • • ? ^im)) 

where the sum is taken over all permutations ii, . . . ,im of the numbers 1, . . . , m, 
then we reduce the representation in ([3]) to that in (jlj). 

Notice also that any [/-statistic is represented as a finite linear combination of 
canonical [/-statistics of orders from 1 to m (called a Hofjding decomposition, see [9]). 
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This fact allows us to reduce an asymptotic analysis of arbitrary [/-statistics to that 
for canonical ones. 

If the distribution F is arbitrary (in particular, it contains atoms) that U- 
statistics in ([3]) or (jlj) admit recurrent representations as linear combinations of 
canonical \^-statistics and [/-statistics of smaller orders. In other words, we can 
represent any canonical [/-statistic as a linear combination of canonical l^-statistics. 
This is a key remark to study the limit behavior of the distributions of [/-statistics. 
In this connection, note an important role of the ^/-statistics with splitting kernels: 

/(tl, . . . = hi{ti)h2{t2) ■ ■ ■ hmitm) (5) 

since, in this case, the corresponding ^/-statistic is represented in the form 

in 1 

i=i i=i 

where Khk{Xi) = and E/i|(Xi) < oo. So, under some dependency conditions of 
the random variables {Xj}, one can apply the multivariate central limit theorem 
which describes the weak limit of the l^-statistic under consideration as the prod- 
uct IVjLiPj^ where {pj] j < m} are centered Gaussian random variables with the 
covariance matrix 

oo 

Ep,p, = E/i,(Xi)/i,(Xi) + (e/i,(Xi)/i,(X,+i) + Eh,{X^)hiiXk+^)) 

k=l 

provided that the series on the right-hand side of this equality exists. It is clear 
that these arguments are not essentially changed when the kernel of a l^-statistic 
can be represented as a linear combination (maybe infinite) of splitting kernels (see 
the proof of Theorem 2). The same argument may be applied to [/-statistics with 
such kernels due to the above-mentioned connection of U- and ^/-statistics. This 
approach was applied to prove some well-known results on asymptotic analysis of 
[/-statistics of independent observations {Xj}. 

In this connection, recall some classic results connected with expansion of a 
canonical function into a multiple orthogonal series with respect to an orthogonal 
basis of the Hilbert space L2(X, F). Since the distribution F has a countable basis, 
the Hilbert space L2(X, F) is separable. It means that, in this space, there exists a 
countable orthonormal basis. Put eo(t) = 1. Using the Gram — Schmidt orthog- 
onalization [S], one can construct an orthonormal basis in L2{X,F) containing the 
constant function eo(t). Denote by such basis. Then Eej(Xi) = for 

every i > 1 due to the orthogonality of all the other basis elements to the func- 
tion eo(t). The normalizing condition means that Ee^(Xi) = 1 for alH > 1. Notice 
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that the collection of functions 

{eii(ti)ei2(t2) ■ ■ ■ ei^{tm)] Zl, ^2, • • • , «m = 0, 1, . . . } 

is an orthonormal basis in the Hilbert space L2(j£'", F"^) (for example, see [8]). 

Thus, one can represent the kernel f{ti,..., tm) of the statistics under consider- 
ation as a multiple orthogonal series in L2(X'", F^' 



7m\ 



f{ti,...,tm) = fii,...,im(^hitl)--- (^imitm), (6) 



where the series on the right-hand side of equality (E]) converges in the norm of L2(X'", F"^). 
Moreover, if the coefficients are absolutely summable then, due to the 

B. Levi theorem and the simple estimate Eejj(X^)-- - ej^(Xj^) < 1, the series 
in ([6]) converges almost surely with respect to the distribution F"^ of the vector 

Consider the case m = 2 and the integral linear operator with a symmetric kernel 
/ G L2(X^, F^) mapping the space L2(X, F) into itself. Since this linear operator is 
completely continuous and self-conjugate, in the separable Hilbert space L2(X, F), 
there exists an orthonormal basis consisting of eigenvectors of this integral operator 
and, for this basis, representation ([6]) for m = 2 is valid. Multiply by an arbitrary 
element €^(^2) the both sides of and integrate these modified parts with respect 
to the distribution F{dt2). Taking orthogonality of the basis elements into account 
we obtain the new identity 

00 

hekih) = fi,kei{ti), 

i=0 

where is the corresponding eigenvalue. From here it immediately follows that fk,k = 
Afc and fi^k = for i ^ k. Therefore, for this basis in the case m = 2, formula ([6]) 
has the form 

00 

f{h,t2) = Y ^kek{ti)ek{t2), (7) 

fe=0 

which was repeatedly employed by many authors. 

Notice also the following property of canonical kernels. 

Proposition 1. If f{ti, . . . ,tm) is a canonical kernel then eo(t) is absent in expan- 
sion i. e., expansion (0) has the form 

00 

f{ti,...,tm) = Y /n,...,i^ei,(ti)--- ei^(t„). (8) 

ii,...,im=l 
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Proof. Denote 

N 

fl^iti, ...,tm) = E . . . ej,„(tm)- 

il,...,im=0 

It is clear that 

N 

il,...,jm = 

Taking degeneracy condition ([1]) into account, we obtain, as ^ cxd. 

In other words, as X — > oo, 

/ ^ \ 2 

IE( E /n,...,jfc-i,o,ifc+i,-,imeii(Xi') . . . eii(X^_;^)ej,(X^+i), . . . , ei^{X^ 

^ jl,...,jm=0 

iV 

= V f 2 ^ n 

^Jl,...,lfc_l,0,2i,4.l,...,Jm 

jl,...,jm=0 

due to the orthonormality of the basis functions. The last relation means that all 
the coefficients /ii,...,ij._i,o,ifc+i,...,i™ vanish for all = 1, . . . , m and all the subscripts 

□ 

Notice also that if the kernel /(ti,t2) in © is canonical then the constant func- 
tion eo(t) is the eigenfunction corresponding to the eigenvalue Aq = of the integral 
operator. So, in this case, the summation in ([7]) starts with k = 1. 

Thus, after replacement of the vector (ti, . . . by the independent observa- 
tions (X^, . . . ,Xj^), the partial sums of the series on the right-hand side of ([8]) {oi 

of ([7]) in the case m = 2^ mean-square converge to the random variable f{X^, . . . , X^) 
and hence they converge in distribution. However, in the present paper, we deal with 
dependent observations for which this property in general is not valid. 



2 The main results for weakly dependent obser- 
vations 

In the present paper, we study stationary sequences {X^} satisfying certain mixing 
conditions. Recall the definitions of the most popular mixing conditions. For j < k, 
denote by Tl^ the a-field of all events generated by the random variables Xj, . . . , X^. 
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Definition 3. A sequence Xi,X2, . . . satisfies a-mixing (or strong mixing) if 

as i 



sup sup 



oo. 



Definition 4. A sequence Xi, X2 
ing) zf 



ip{i) := sup sup 

fc>i AeOTj,BeOT^.,P{A)>o 

Definition 5. A sequence Xi, X2, . . . 

ip{i) := sup sup 

fc>i AeS!n5=,-BG9n^.,p(A)P(B)>o 



F{AB) - P(A)P(5) 
. . . satisfies ip-mixing (or uniformly strong mix- 

¥{AB) - f{A)¥{B) 



P(A) 

satisfies ip-mixing if 
P(A5) - P(A)P(5) 



as i — > 00. 



P(A)P(5) 



as i ^ cxD. 



It is clear that the sequences and |'?/'(«)| are nondecreasing and 

'0-mixing is stronger than (/3-mixing which in turn imphes a-mixing. 



In the sequel, in the case of 99-mixing, we assume that 



Y.V^'^{k) < 00. 

k=l 



(9) 



Note that this known condition provides the cental limit theorem for the correspond- 
ing stationary sequences (for example, see [1]). 

Introduce also the following restriction on finite-dimensional distributions of the 
stationary sequence {Xi}. 



(AC) For every collection of pairwise distinct subscripts (ji 



) Jr. 



the distri- 



bution of (Xjj, . . . , Xj^) is absolutely continuous with respect to the distribution of 
{XI, . . . ,X^). 

Notice that this restriction will be nontrivial only for sequences under a- or y)- 
mixing because, in the case of ^/'-mixing, by induction on m, from Definition 5 we 
can easily deduce the inequality 

m 

P{X,, e Ai, . . . , x,^ e Am) < (1 + ^(1))" n Pi^k e A^) 

k=l 

which is valid for every collection of Borel subsets {Ai, Am) and for every pairwise 
distinct subscripts (ji, . . . ,jm)- From here condition (AC) immediately follows. 
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Remark 1. As was mention before, the condition 



oo 



\fiu-,iJ<^ 



(10) 



implies convergence of the series in ^ almost surely with respect to the distribution 
of the vector {X^,...,X^). So, under condition (AC), this convergence is valid 
almost surely with respect to the distribution of the random vector {Xj-^, . . . ,Xj^). 
In other words, if condition (AC) is fulfilled then, for any pairwise distinct subscripts 
ji, . . . ,jm, we can substitute the random variables Xj^, . . . ,Xj^ forti, . . . ,tm 

Remark 2. Under restriction [T^) . one can sometimes obtain the above-mentioned 
multiple series expansions without any restrictions like (AC) on joint distributions 
of the initial stationary sequence. For example, if the kernel f(ti,..., tm) is contin- 
uous in M*" and all the basis functions ek{t) are continuous and bounded uniformly 
in k then, under condition fTU\). equality ^ is transformed into the identity in 
(see the proof of Theorem 2 below). Therefore, in this identity, one can substitute 
arbitrarily dependent random variables Xj-^, . . . , Xj^ (in particular, for coinciding 
subscripts jk) for the arguments ti, . . . ,tm. 

As an example of expansion ([8]) which is everywhere vahd, consider the following 
symmetric kernel canonical with respect to the [—1, l]-uniform distribution: 



where W{t) is a standard Wiener process on the positive half-line. The eigenval- 
ues of the corresponding covariance operator, i. e., of the integral operator with 
the kernel f*{t,s) in the Hilbert space L2[— 1,1], are calculated by the same for- 
mula as that for the eigenvalues of the covariance operator of a standard Wiener 
process: = (n{k — 1/2)^ , k = 1,2,... (for example, see [5l . 8.6]). The 
corresponding eigenfunctions in ([8]) which form an orthonormal basis (together 
with the constant function eo{t) = 1) in L2[— 1,1] are calculated by the formula 



It is easy to see that this example satisfies the conditions of Remark 2. So, for 
arbitrarily dependent random variables Xi and X2 with the [—1, l]-uniform distri- 
bution of each Xj, the random variable f*{Xi,X2) can be represented in such a 
way: 




This is the covariance 



function of the Gaussian process sign(t)M^f |t| j defined on [—1, 1], 






(11) 
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and the series absolutely converges everywhere. 

Under y?- mixing for the sequence {Xi} but without the restrictions mentioned 
in Remarks 1 and 2, in general one cannot use expansions (I7l) or ([8]). Such mistake 
is contained in [1] (see also [9j), where it is claimed that, in the case of yj-mixing 
stationary observations for m = 2, under condition ([9]) only but without any restric- 
tions like (AC) and the regularity condition mentioned in Remark 2, the following 
assertion is valid: 

oo 

(12) 

fc=i 

where {A^} are the eigenvalues of the integral operator with the symmetric ker- 
nel f{ti,t2), which are assumed to be summable (^i. e., under condition (10)^, and 
{rfc} is a Gaussian sequence of centered random variables with the covariances 

oo 

Enn = Ee,,(Xi)eKXi) + J2 [EekiX,)ei{Xj+,) + Eei{X,)ekiXj+,)] , (13) 

i=i 

where |eA;(t)| are the eigenfunctions corresponding to the eigenvalues {A^} and 
forming an orthonormal basis in L2{X, F) (factually, the first summand on the right- 
hand side f|T3l) is the Kronecker symbol 6k,ij- In (12) and in the sequel, we admit 
degeneracy of some random variables Xj. In other words, we add to the class of Gaus- 
sian distributions the all weak limits when the variance tends to zero. Actually, the 
similar agreement is contained in p^J. To prove relation (12) in [1] the author use 
the expansion of the kernel /(ti,t2) in series ([7]) with respect to the basis |efc(t)|. 
Due to the above-mentioned arguments, we could now substitute the independent 
observations X* and X* for the variables ti and t2- The same is true for a pair Xj 
and Xj from a stationary sequence satisfying condition (AC). However, in [1], the 
author substituted a pair Xi and Xj from an arbitrary stationary sequence under ip- 
mixing with restriction (Q only. But under this replacement, the above-mentioned 
equalities may be not fulfilled with a nonzero probability. Moreover, in this case, 
the limit law in ( |T2i) may change the form. The idea of constructing examples of 
such a kind is very simple: We need to construct a stationary sequence {Xi} with 
a non-atomic marginal distribution, such that its elements Xj and Xj coincide with 
nonzero probability for some subscripts i ^ j- We then can change the values of / 
on diagonal subspaces to break the above-mentioned identities with a nonzero prob- 
ability when we replace the arguments X* and X* of the kernel with the dependent 
pair Xi and Xj. The corresponding construction is contained in the proof of the 
following assertion. 

Proposition 2. There exist a stationary sequence {Xi} and a canonical kernel /(ti, ^2) 
satisfying all the restrictions in However, under substituting the observations Xi 
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and X2 for ti and t2 respectively, the series in does not coincide with the kernel. 
Moreover, the weak limit for the distributions of the U -statistics differs from 

So, under certain conditions (^say, conditions ffTOl) and (AC)^, fZ-statistic ([3]) can 
be represented as the following multiple series converging almost surely: 

00 

il,...,im = l l<jl7^---^jm<™ 

Further analysis is similar to that in the i.i.d. case, i. e., it is reduced to extraction 
of ^-statistics with splitting kernels from the multiple sum on the right-hand side 
of this identity. The main fragment of the proof in [TT] is as follows: The value 



is represented as a linear combination of products of the values 

n \ ^ \ ^ 

The proof has a combinatorial character and does not depend of joint distributions 
of the random variables {Xj}. Further we apply the corresponding laws of large 
numbers as well as the central limit theorem and the following simple assertion. 

Proposition 3. Let x G M'^, y G W , he a continuous function. Let {C„} he 

an arhitrary sequence of random vectors in M'^ weakly converging to some random 
vector (. Let {rjn} he a sequence of random vectors in W defined on a common 
prohahility space with {Cn}, which converge in prohahility to a constant vector cq. 
Then the following weak convergence is valid: 

HCn,Vn) ^ $(C,Co). 

In the present paper, ^{x, y) is a polinomial of components of vectors x and y, 
and the sequence (n is defined by the formula 



Cn:= |n-V2^ei(X,),...,n-V2^e;v(X 
and 7]n is the finite collection 



j=i i=i 



n 



-k/2 



^Ci^iXj) ■ ■ ■ Ci^iXj); 2 <k <m, ii,...,ik< N 
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here is an arbitrary natural number. 

So, under the condition /(ti, . . . , tm) G I/2(X'", -F™), in the case of i.i.d. random 
variables {Xj}, it was proved in [llj that 

oo oo 

Un^ J2 n (14) 

h,...,im=l j=l 

where {xj} is a sequence of independent random variables having the standard Gaus- 
sian distribution, . . . , i^) := J2T=iSj,iry and Hk{x) are Hermite polynomials 
defined by the formula 

H,(.) := (-l,VV^|j(e-'V=) 

or by the recurrent formula 

Hq{x) = 1, Hi{x) = X, 

Hn+l{x) = xHn{x) - nHn-l{x). 

Thus, Hk{x) is a polynomial of degree k and the product on the right-hand side 
of ffT^ can be represented in the form 

max{ife} 

n = HrArn) ' " " ^..(^.J, (15) 

j=mm{jfe} 

where the natural numbers and js are defined by the relation = Y^^=i5js,iri 
and at that, Z^ksH = ^ ^"^^ niin{zfc} < ji < max{ifc} for all / < s. Therefore, the 
right-hand of (fTSl) is a polynomial of degree m of the variables r,^ , . . . , tj^ and with 
coefficients having a universal upper bound depending on m only. 

The goal of the present paper is to prove limit representations of the form (fT^ 
in the case of weakly dependent random variables {Xj}. 

Introduce some additional restrictions on the mixing coefficients and the basis 
functions in the expansion in (1141) . In the sequel, we assume that the stationary 
sequence {Xi} satisfies either a-mixing or mixing, and moreover, the ortho normal 
basis {ej(t)} in L2(X, F) with the original element eo(t) = 1, satisfies the following 
additional restrictions: 

1. In the case of (^-mixing, we assume condition to be satisfied and 

supEej(Xi) < OO] (16) 
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2. In the case of a-mixing, we assume that, for some e > and an even num- 
ber c > m, 

^ m+e 

supEei(Xi) < oo, (17) 

i 

oo 

J2k'-^a'/^'^'\k) <oo. (18) 

k=l 

Further, introduce a sequence of Gaussian centered random variables {Ti} with 
the covariances 

oo 

ErkTi = Eek{X,)eiiXi) + ^ [Eek{X,)ei{X^+,) + EeiiX,)ek{X^+,)] . (19) 

i=i 

It is easy to see that, due to orthonormality of the basis {cfc}, we have 



Er2 = 1 + 2 ^ Eefc(Xi)efe(X,+i), 

oo 

ETfeTi = J2 [Eefc(Xi)eKX,-+i) + EeKXi)efc(X,-+i)] , k ^ I. 
i=i 

The existence of the series in flTIJl) follows from the above-mentioned restrictions on 
the mixing coefficients, and it will be proved later. In what follows, the Gaussian 
sequence {xj} will play a role of the weak limit as n — oo for the sequence 



n 



-V2;^ei(X,), n-V2;^e2(X,),... 



The main results of the present paper are contained in the following two theo- 
rems. 

Theorem 1. Let one of the following two conditions be fulfilled: 

1. The stationary sequence {Xj} satisfies ip-mixing, (0), and p6l) : 

2. The stationary sequence {Xj} satisfies a-mixing, ( | iTp , and i\lS\) . 

Then, for any canonical kernel f(ti, . . . , tm) G L2(X'^, F™), under conditions pOll 
and (AC), the following assertion holds: 

oo oo 

Unif)^ E /n,....™n^-.(n,....,„)(^.), (20) 
n,...,im=i i=i 

where Un{f) is a statistic of the form (0) and the centered Gaussian sequence {tj} 
has the covariance matrix defined in i\19[) . 



11 



Theorem 2. Let X be a separable metric space and let a canonical kernel f{ti, . . . , tm) 
be continuous (in every argument) everywhere on X™, and let it satisfy f 1 1 C\) . More- 
over, if all the basis functions ek{t) in (E]) are continuous and uniformly bounded 
in t and k, and one of the two conditions of Theorem 1 is valid then, as n ^ oo, 

oo 

Vnif) ^ J2 fiu-,in.^h ■ ■ ■ ^in., (21) 

il,.--,im=l 

where the Gaussian sequence {xj} is defined in Theorem 1. 

Remark 3. It is known that, in the i.i.d. case, condition / flOj) of absolutely summa- 
bility of the coefficients in the series expansion of the kernel can be weakened up to 
summability of the coefficients squared f77|, [^. In the same time, in limit theorems 
for the corresponding V -statistics, the latter condition does not describe the limit 
behavior since to define the weak limit, we need the existence of moments of the ker- 
nel on all the diagonal subspaces. For example, under the regularity conditions only 
(without (10)) of Theorem 2 for bivariate V -statistics, the assumption of finiteness 
ofK f{Xi,Xi) is equivalent to summability of the sequence Xk = fk,k in represen- 
tation say, if all Xk are positive. However, in the i.i.d. case, for the kernels of 
a bigger order, we need no summability of the coefficients fi^,...,i^ on the set of all 
pairwise distinct subscripts. 

As is noted in Proposition 4 below, in the case of dependent observations, we can- 
not omit the above-mentioned restriction regarding summabihty of the coefficients 
fii,...,im oil the diagonal subspaces for [/-statistics as well. 

Proposition 4. There exist a stationary 1-dependent sequence {Xi} satisfying con- 
dition (AC), and a canonical kernel f{ti,t2) G L2(X^,F^) such that the weak limit 
of the corresponding U -statistics does not exist. 

Remark 4. In ^2], in the case of dependent observations, another approach was 
proposed for description of the limit distribution of canonical Von Mises statistics 
as a multiple stochastic integral of the kernel under consideration, with respect to 
increments of a centered Gaussian process with a covariance function defined by joint 
distributions of the random variables {Xi). In the i.i.d. case, such dual description 
of the limit low is well known (for example, see J^j. 

However, the stationary sequence {Xi] in [2j must satisfy a stronger T/^-mixing 
condition. In the same time, in contrast to the present paper, condition (fTOl) and 
the regularity conditions for the kernel and the basis functions of Theorem 2 were 
replaced in [2] with the more natural condition of integrability of the kernel on all the 



12 



diagonal subspaces. Note that the above-mentioned regularity condition and (fTOj) 
imply the boundedness of the kernel under consideration, i. e., the above-mentioned 
condition of the kernel integrability on the diagonal subspaces in [2] is fulfilled. 

It is not clear importance of restriction ( JTOl) outside the diagonal subspaces 
(i. e., on the set of all pairwise distinct subscripts) to approximate U- and V^-statistics 
of an arbitrary order for dependent trials. 

Mention also the important particular case when the bivariate kernel of a V- 
statistic is represented as an inner product f{x,y) = {x,y) in a separable Hilbert 
space. In this case, the corresponding Von Mises statistic coincides with the Eu- 
clidian norm squared of a normalized sum of weakly dependent centered observa- 
tions and we deal with the Central Limit Theorem (with respect to the class of all 
centered balls) for Hilbert-space-valued weakly dependent observation which was 
proved under various mixing conditions and the existence of the moment IE(Xi, Xi) 
(for example, see p[2]). 



3 Proof of Theorems 1 and 2 and Propositions 2 
and 4 

First of all, we formulate the following two auxiliary assertions which are versions of 
the classical Rosenthal moment inequality for sums of independent random variables. 

Theorem 3 (A [I3]). Let C,i be a sequence of centered random variables having 
finite moments of order t > 2 and satisfying (f -mixing with the restriction ip : = 
Y.T=i </'"^^^(2'^) < oo. Then, for t > 2, the following inequality holds: 



E max \Sk\^ < (tc((p)'* 

l<fc<n 



( n ( n \ 



i=l \ 1=1 



where the constant c{(f) depends on ip only. 

Theorem 4 ( [3J). Let S,i be a sequence of centered random variables with finite 
moments of order t > 2 satisfying a-mixing and let, for some e > and even c> t, 
the following condition be fulfilled: 

oo 

E A;^-V/("+")(A;) < oo. 

k=l 

Then there exists a constant C depending on t and the mixing coefficient a{k), such 
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that 



ElSrf < Cmax 



/ / \ 



E(E|&|'«)'"-'. E(EI&I 

k=l \ k=l 



Proof of Theorem 1 . Consider the following partial sum for the above-mentioned 
multiple series expansion of the kernel f{ti,..., tm)'- 

fN{tl, ■ ■ ■ ,tm) '■= X! fii,...,im<^ii{'tl) ■ ■ ■ <^im{'tm)- 

l<il,...,im<N 

It is clear that, due to linearity of the functional Un{-), we have 

UnifN) = fh,...,imUniei, ■ ■ ■ dj, 

l<ii,...,im<N 

where 

Unie^, ■ ■ ■ := n'^/^ J2 e,,(X,J ■ ■ ■ e,jX,J. 

Further, by analogy with the above-mentioned arguments for independent observa- 
tions, we represent the [/-statistic as a sum of Von Mises statistics in which of them 
the summation is taken over all the subscripts (not necessarily pairwise distinct). 
Further, changing the order of summation, we start to study the random variables 

-j^ n \ ^ 1 " 

j=i j=i j=i 

For each k > 2, these sums converge to zero in probability as n — oo since, by 
condition f|T6l) . there exists the finite moment E ej^(Xj)-- - ei^(Xj) . Hence, by 
the law of large numbers for weakly dependent random variables, the following 
convergence is valid: 

1 

- (Xj) ■■■ (Xj) Ecii i^j) ■■■ <^ik i^j) ill probability. 

n j=i 

So, for k > 2, we obtain 
1 " 

Y en(Xj) ■ ■ ■ ei^{Xj) ^ in probability. 



Therefore, the summands containing such sums as factors, converge to zero in prob- 
ability as well. For = 2, we also apply the law of large numbers to the sums 
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mentioned. Due to orthonormality of the basis, this hmiting values coincide with 
the Kronecker symbol Therefore, the limiting form of the partial sum is quite 
similar to that in the i.i.d. case: 



UnU 



N) 



fii,...,im rr -^'^jih,..., 



VooN, 



l<ii,...,im.<7V 



where the random variables {rfc} have joint Gaussian distributions with the covari- 
ance matrix defined in ( |T9l) . The last assertion is a consequence of the multivari- 
ate Central Limit Theorem for finite-dimensional projections of empirical processes 
based on stationary observations and indexed by the family of functions {cfc}. In 
turn, the last result follows from the univariate CLT for stationary sequences and 
from the Cramer - Wold method to reduce the multivariate case to the univariate 
one [H Theorem 20.1]. 

Further, using the well-known covariance inequalities for stationary sequences, 
we prove finiteness of the covariance we need. 

(a) For yj-mixing, from [3, 17.2.3] and condition iQ it follows that 



\^rkTi\< Ee,.(Xi)e/(Xi)| + E |Eefc(Xi)eKX,+i; 

oo 



+ 



Eei{Xi)ek{Xj, 



(b) For a-mixing, due to [7, 17.2.2] and condition f|T8l) . we have 



\ErkTi\ < \Eek{Xi)ei{Xi)\ + J2 



+ 



Eei{Xi)ek{Xj. 



<6k,i + CY.a^/^'+^\j) <oo. 



Notice that, in cases (a) and (b), the covariances Et^ti uniformly bounded. 
We now represent the prelimit and limit statistics in such a way: 

f/n(/) = f/n(/iv) + f/n(/-/iv), 
Voo = ''7ooAf + V<xN, 



where 



'?ooAf 



E/ fil,...,im rr -^l^j(n v,«m) (T?) • 
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Prove that, as — > oo, the random variables Un{f — /n) and rjooN converge to 
zero in mean. It is clear that 



E 



max{jfc}>A'' 



Uj(il,...,im)\Jj 



where, as was noted before, YYfLiHuj{i^,...,i^){Tj) is a polynomial of order m of the 
variables tj^^, . . . , tj^ (see the notation in (fTS!) ^ , and all the coefficients of the polyno- 
mial depending on the multiplicity ri of subscripts in a fixed collection {ii, . . . ,im), 
have a universal upper bound depending only on m. The absolute moment of each 
monomial of this polynomial is bounded uniformly in . . . , i^- It suffices to show 
this property for the high-order member of the polynomial. Instead, by Holder's 
inequality and relation X^ks f'l = ''^j we have 



E 



'31 



< Elr, 



ji I 



Ti/m 



■ Elr, 



Vs/m 



Since the variances of the limit Gaussian random variables Tj are uniformly bounded, 
their absolute moments of order m are uniformly bounded as well. Hence, the 

is bounded uniformly over all Zi, . . . , im- Therefore, 



moment E 



condition (ITO!) implies the limit relation E 



VooN 



as iV — >• 00. 



Consider now the series tail in the expansion of [/-statistic. We have 



E 



Unif - In) 



n 



n 



n 



E (/(^ii ' • • • ' ^im) " fN{Xj^ , . . . , Xj^ 

E E /n,... 

(XjJ ■ ■ ■ ei„(Xj^; 

E/ $Z fil,...,irn.^il{-^jl) ' ' ' ^imi-^jm, 

max{ife}>Af l<jij^--- T^jm<n 



< n 



max{is;}>Af 



max{jj.}>A'' 



Consider the value 



3m / ■ 



(22) 
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Due to summability of the coefficients it suffices to prove the uniform 

boundedness over all ii, . . . ,im of the value in (122|) . As before, we add and subtract 
into the multiple sum in (|22l) summations over diagonal subspaces obtaining a linear 
combination of monomials of the form 



X 



(X,)---e,„(X, 



^mi/2 yirrik/'i ' 

where (/i, . . . , Im) is a permutation of the multiindex (ii, . . . , im) and the natural 
number mi satisfies the relation mi + ■ ■ ■ + = m, k < m, and the number of 
such monomials is bounded by a number depending on m only. Thus, proving the 
uniform boundedness of the absolute moment of such monomials we now prove the 
uniform boundedness of the absolute moment of the value in fl22l) . We have 



E 



EUei,{X,)...ei,^^{X,) 



n 



mi/2 



X 



^mfc/2 



<n 



E 



V 



n 



mr/2 



m/mr \ ''"'■/™ 



where mg = 0. Prove that each of the factors above is uniformly bounded. Without 
loss of generality, one can verify this only for the ffist factor. Since in general the 
summands ei^{Xj) ■ ■ ■ ci^^ (Xj) are not centered, we consider separately the following 
two cases. 

First, let mi = 1. In this case, we estimate moments of a sum of centered weakly 
dependent random variables. Under (/9-mixing, we use Theorem A. Then 



E 



< Co(m) 



1 



EE(e.(X,))' 

< Co(m)(l +n-('"/2+i)^(^)) 
<Co(m)(l+C(m)). 
Under a- mixing, by Theorem B, we obtain 



m/2 



E 



<Co(m) Ee,(X 



n 



2+e\ 2/(2+£) 



m/2 



+ Co(mK™/2^(Ee,(X,: 
< CAm). 



m+e\ in/im+e) 
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Now, let mi > 2. Then, by the arithmetical convexity inequality for sums and 
by Holder's inequality, we obtain the following simple estimate: 



E 



n 



mi/2 



m/m\ 



< n™/"^in-"^/2E|e^,(Xi) ■ ■ ■ (Xi)P^"' < sup E|e,(Xi 



m 



Thus, 

Theorem 1 is proved. 



as ^ oo. 



□ 



Proof of Theorem 2. Without loss of generality, we assume that the separable metric 
space X is the support of the distribution F. It means that the space X does 
not contain open balls having zero F- measure. Since all the basis functions ek{t) 
in ([H]) are continuous and bounded uniformly in t and by the Lebesgue theorem 
on dominated convergence, these above facts imply continuity of the series in ([8]) 
under condition ffTOl) . It is easy to see that, in this case, the equality in ([8]) turns 
into the identity of the variables ti, . . . ,1^ since equality of two functions on an 
everywhere dense subset of X implies their coincidence everywhere in X. Therefore, 
we can replace the variables ti,...,tm in identity ([H]) with arbitrarily dependent 
observations, in particular, coincident ones. This is a principal distinction from the 
proof of the previous theorem. Hence, for the all elementary events, the following 
representation holds: 



Vnif) 



n 



-1/2 



n 



-1/2 



Further arguments actually repeat the arguments (even in a simpler version) of the 
proof of Theorem 1. Theorem 2 is proved. □ 

Proof of Proposition 2. Let {Yi] i > 1} be a sequence of independent random vari- 
ables uniformly distributed on [—1, 1], and let {^j; i > 1} be a sequence of inde- 
pendent symmetric Bernoulli random variables which are independent of {Yi} as 
well. Set Xi = Yi+^^. The random variables {Xi} form a stationary 1-dependent se- 
quence. Notice that, in this case, the random variables Xi are uniformly distributed 
on [—1, 1] as well. Thus, the stationary sequence {Xj} satisfies y)-mixing condi- 
tion and the restriction in ([2]). It is clear that, for the corresponding independent 
copies X^ and X^, we have P{X^ = Xj) = 0, but for the originals, we obtain 



p(Xi = X2) = P(6 = i)P(6 = 0) = 1/4. 
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Notice that, in the example under consideration, the basis functions and the coeffi- 
cients in the series expansion in ([8]) do not depend on the values of the kernel / on 
the diagonal due to continuity of the marginal distribution. Let 



sup 

\tl\,\t2\<l 



f{tl,t2 



< 1. 



Now, change the diagonal values setting f(t,t) = 1 + (3, for all t G [—1,1], where P > 
0. Then P(^f{Xi, X2) = 1 + /?^ =1/4. In the same time, the series on the right-hand 
side of (IH]) does not depend on (3. 

It is easy to show that, in the example under consideration, the limit law will 
essentially differs from f|T2|) . Instead, represent the bivariate [/-statistic with the 
symmetric canonical kernel /(ti,t2) in the form 

Un = -Y.fiX^^Xi+l)+Un. (23) 

n ~r 

Limit behavior of the ffist sum on the right-hand side fl23l) is simply studied since 
the random variables Xj+i); i > l| form 2-dependent stationary sequence for 

which the strong law of large numbers is valid. Hence, as n — > 00, the limit relation 

holds with probability 1. Under the above-mentioned restrictions on the kernel, the 
limit equals ^-j^. 

The statistic Un is based on the random variables f{Xi, Xj) under the condition 
|z — j| > 2 which provides independence of the random variables Xi and Xj. Since 
the distribution of Un (hence, and the limit one if it exists) does not depend on /3, 
this proves the assertion. 

We now compute the above-mentioned limit law for the sequence {Un} under 
absolutely summability of the sequence {Afe} in the series expansion in ([7]) for the 
kernel /(ti,t2) under consideration. Notice that the kernel with the indicated prop- 
erties exists (see the example after Remark 2 in the previous section). We note once 
again that, in the example under consideration, all the eigenvalues and all the 
basis functions Ckit) do not depend on the diagonal values of the kernel (i. e., do 
not depend on j3 under the additional above-mentioned restrictions on the kernel). 
Since the statistic f/„ consists only of the random values f{Xi,Xj) under the con- 
dition |« — j| > 2 which provides independence of the random variables Xi and Xj, 
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we can replace f{Xi,Xj) with the corresponding double series from ([7]). Then 
Un = - E hekiXi)ek{Xj) 

^ i,j<n: \i-j\>2 k>l 



-E^^-(^'-3/2), (25) 



' 2 1 

-5:e,(X,)efc(X,+i)--E4W 

^ i<n ^ i<n 



k>l 



where {rk} is a centered Gaussian sequence with covariances ([T9|l . To compute 
the weak limit as n ^ cxd, we use, first, the fact of the weak convergence of 
the collection of sequences (in other words, their finite-dimensional distributions) 
[:^J2i<n^kiXi); > l| to the Gaussian limit {r^; k > 1}, second, the laws 
of large numbers both for the 1-dependent and 2-dependent stationary sequences 
{e|(Xj); i > 1} and |efc(Xj)efe(Xj+i); i > l| respectively, taking into account 
Proposition 3 as well as the relations Ee|(Xi) = 1 and Eefc(Xi)efc(X2) = 1/4 for 
all > 1. Moreover, we used simple upper bounds for the absolute moments of 
remainders of the series in fl25|) . So, from fl23l) - fl25l) we obtain the form of the limit 
law: 

^ Ef{Y„ Y,)/2 + ^ A, (r^ - 3/2) . 

The right-hand side coincides with (fT2l) . say, under the conditions of Remark 2 (or 
of Theorem 2) but does not coincide under the above-mentioned restrictions on 
diagonal values of the kernel /. Proposition 2 is proved. □ 

Proof of Proposition 4. Define the 1-dependent stationary sequence {Xi} by the 
scheme of Proposition 2 using i.i.d. random variables {1^} with the following distri- 
bution (different from that in Proposition 2): 



k with probability pk = 2 ^ ^, 
-k with probability pk = 2^^^^ 



for all natural k. We assume that X coincides with the support of the distribution F, 
i. e., with the set of all nonzero integers. It is easy to verify that the stationary 
sequence {Xi} satisfies condition (AC) since, under an arbitrary dependence of the 
components of the vector (Xj^, . . . , Xj^), the support of its distribution is contained 
in the support of the m-variate distribution of (X*, . . . , X^). 
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Consider in L2(X, F) the sequence of functions 

Co = 1; 

r 2'=/2 if t = k; 
ek{t) = l-2^/^ iit = -k; 

[ otherwise. 

It is clear that this sequence is an orthonormal basis in the space L2{X,F). Let 
{Xk}k>i be an arbitrary sequence of positive numbers summable squared. Define a 
symmetric canonical kernel f{t, s) by the formula 

oo 
k=l 

It is clear that 

r A|,|2l'=l ifA; = /; 
/(fc,0= -A|fc|2l'=l iffc = -/; 

[ otherwise. 

Moreover, {Xk} and |efe(t)| are the respective eigenvalues and eigenfunctions of the 
integral operator with the kernel f{t, s). 

We now employ the expansion in (23). Using the same arguments as in (25), we 
obtain 

oo 

Un^j:^kiri-3/2), (26) 

k=l 

where {t}.} is a Gaussian sequence with covariances (fT9|) which, in the case under 
consideration, are calculated in such a way: ErfcT/ = if 7^ /, and Er| = 3/2, i. e., 
the limit sequence consists of independent centered Gaussian random variables with 
the common variance 3/2. Therefore, by the Kolmogorov Three Series theorem (for 
example, see [5]), the series on the right-hand side of fl26|) converges almost surely; 
we need only summability squared of the sequence {A^} at that. Consider in detail 
the limit behavior of the first sum in expansion (23). To transform this sum, use 
the following simple identity: 

/(Xj, Xj+i) = ^j(l — ^i+i)/(Fj+i, Fi+i) + ^j^j+i/(Kj+i, Fi+2) 

+ (1 - m+lfiy^, y^+2) + (i - m - ^^+l)fiy^, y^+l)■ 
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Every term of the fourth summands on the right-hand side of this identity is the 
ith member of the corresponding 1- or 2-dependent stationary sequence. Hence, for 
each partial sum consisting of these summands except the first one, the strong law 
of large numbers is applicable and the corresponding normalized sums tend to zero 
almost surely due to degeneracy of the kernel. However, the law of large numbers is 
not applicable to the sum consisting of the summands Vi :— ^,(1 — l^+i) 
since the expectation Ez/i = ^ J2k ^fe ^i^Y be infinite. In this case, the limit behavior 
of the first sum on the right-hand side of (23) is described by the following 

Lemma 1. Let {fik} be a sequence of i.i.d. nonnegative random variables with 
infinite expectations. Then 

lim - V /Xfe = oo. 

n— »oo fi ^ — ' 

k<n 

almost surely. 

The proof of this simple assertion is a direct consequence of the Borel — Kan- 
teUi lemma since the condition E/xi = cx) is equivalent to divergence of the series 
J2k P(A*fc > ^k) for an arbitrary fixed > 0. Prom here we immediately conclude 
that ^ 

lim inf — uu > N 

almost surely which was to be proved. 

Notice that the requirement of nonnegativity of the summands is essential. The 
corresponding counterexample is constructed by i.i.d. random variables Hk with the 
Cauchy distribution. 

Since the nonnegative identically distributed random variables {i^k} form a 1- 
dependent stationary sequence, splitting the corresponding partial sum into two ones 
(each of them consists of independent summands), we easily reduce the problem to 
that described above. 

Thus, in the case under consideration, the existence of a finite weak limit for the 
[/-statistics is equivalent to summability of the coefficients {Xk}- Proposition 4 is 
proved. □ 



ACKNOWLEDGMENTS 

This research was supported by the Russian Poundation for Basic Research 
(grant 06-01-00738) and by the Ministry of Higher Education and Science of the 
Russian Pederation (grant .2.1.1.1379). 



22 



References 



[1] p. Billingsley, Convergence of probability measures (Wiley Series in Probability 
and Math. Statistics, John Wiley, New York, 1968). 

[2] I. S. Borisov and A. A. Bystrov, "Limit theorems for the canonical von Mises 
statistics with dependent data," Sibirsk. Mat. Zh. 47 (6), 1205-1217 (2006) 
[Siberian Math. J. 47 (6), 980-989 (2006)]. 

[3] P. Doukhan, Mixing. Properties and Examples (Lecture Notes in Statistics 85, 
Springer- Verlag, New York, 1994). 

[4] G. K. Eaglcson, "Orthogonal expansions and [/-statistics," Austral. J. Statist. 
21 (3), 221-237 (1979). 

[5] L L Gikhman, A. V. Skorokhod, and M. I. Yadrcnko, Theory of probability and 
mathematical statistics ( Vyshcha shkola, Kiev, 1979) [in Russian] . 

[6] W. Hoeffding, "A class of statistics with asymptotically normal distribution," 
Ann. Math. Statist. 19, 293-325 (1948). 

[7] I. A. Ibragimov and Yu. V. Linnik, Independent and stationary sequences of 
random variables (Nauka, Moscow, 1965) [Noordhoff, Groningen, 1971]. 

[8] A. N. Kolmogorov and S. V. Fomin, Elements of the theory of functions and of 
functional analysis (Second edition, Nauka, Moscow, 1968) [Dover Publications, 
1999]. 

[9] V. S. Korolyuk and Yu. V. Borovskikh, Theory of U-Statistics (Naukova Dumka, 
Kiev, 1989) [Kluwer Academic Publ., Dordrecht, 1994]. 

[10] R. Von Mises, "On the asymptotic distribution of differentiable statistical func- 
tions," Ann. Math. Statist. 18, 309-348 (1947). 

[11] H. Rubin and R. Vitale, "Asymptotic distribution of symmetric statistics," 
Ann. Statist. 8 (1), 165-170 (1980). 

[12] A. N. Tikhomirov, "On the accuracy of normal approximation of the probability 
of hitting a ball of sums of weakly dependent Hilbcrt Space valued random 
variables I," Teor. Veroyatnost. i Primenen. 36 (4), 699-710 (1991) [Theory 
Probab. Appl. 36 (4), 738-751 (1991)]. 



23 



[13] S. A. Utev, "Sums of random variables with (/7-mixing," Trudy Inst. Mat. 13 

Asymptotic analysis of the stochastic process distributions, 78-100 (1989) [Sib. 
Adv. Math 1 (3), 124-155 (1991)]. 



24 



